Speaker Extraction With Co-Speech Gestures Cue

Speaker extraction seeks to extract the clean speech of a target speaker from multi-talker mixture speech. There have been studies use pre-recorded sample or face image as cue. In human communication, co-speech gestures that are naturally timed with also contribute perception. this work, we explore sequence, e.g. hand and body movements, cue for extraction, which could be easily obtained low-resolution video recordings, thus more available than recordings. We propose two networks using perform attentive listening on speaker, one implicitly fuses in process, other performs separation first, followed by explicitly associate separated speaker. The experimental results show is informative associating